The present invention relates to volume rendering, and more particularly to the realistic rendering of volumetric datasets with realistic shading and shadows.
Direct volume rendering (DVR) is a method for obtaining a 2 dimensional (2D) image of a 3 dimensional (3D) dataset. Unlike many other techniques, the DVR technique enables all of the data in the dataset to contribute to the 2D output image.
Volume rendering is often described as direct rendering because it directly maps sample points from the volume to a viewplane via some optical model and a transfer function that controls the extent to which the data values are transparent (or opaque). A variety of DVR methods exist, but most are based around the idea that voxels are assigned a color and a transparency mask. This transparency means that obscured voxels may still contribute to the final image, and it is this mechanism that allows DVR to display an entire 3D dataset, including the internals of an image. An advantage of DVR is that DVR has the potential to directly visualize the complete dataset, in 3D, and with the internals of the 3D image at least partially exposed.
DVR images may be fuzzy and/or blurry because the images are generated by considering many points in the dataset and merging them. As such, DVR images are not particularly suited where crisp images are required to study fine detail. The underlying principle of DVR is sampling the volume dataset along rays in an arbitrary direction and blending these samples together. Before being blended, the densities read in the volume are converted to colors with a transfer function that is often encoded as a lookup table. When a sufficient number of samples are involved, the result is often a seamless image.
There are techniques for obtaining shadows at an interactive rate (i.e., several frames per second) but using the shadows in conjunction with shading cumulates the computational overhead of both techniques. To perform volume rendering with shadows, a commonly used method is to accumulate the light attenuation of each slice of the volume in an off-screen shadow map. A shadow map generates shadows as if light emanated from a point light source.
After the rendering of each slice into a framebuffer, it is rendered a second time into the shadow map with a shader that accumulates the light attenuation. A framebuffer refers to the memory dedicated to storing the image. Generally, the framebuffer is a 2 dimensional (2D) array of pixels, where each pixel stores a color.
At a given time in the rendering process, the shadow map contains the attenuation of the already drawn part of the object seen from the light point of view. The content of the shadow map is used as the shadow information for the subsequent slice rendered into the framebuffer using projective mapping. This process is repeated iteratively for each slice.
In classical rendering models, a distinction is made between two types of light—diffuse light that is the consequence of light striking the surface of the object directly and specular light that appears locally in areas which reflect light towards the eye of the viewer. The final pixel color depends on these two light terms that are computed independently.
The traditional approach to shading in volume rendering involves computing a local gradient per voxel using a central difference derivation. A local gradient determines the changes in a dataset locally with respect to a particular point in the volume. The Blinn-Phong lighting model is also commonly applied to each fragment on the rendered slices. The Blinn-Phong lighting model enables the approximation of shading without being too computation intensive. This gradient-based approach, however, suffers from several problems that affect performance and image quality.
Computing gradients in real-time often requires many texture fetches, causing performance to be bound by memory bandwidth. Precomputing gradients reduces the number of texture fetches but is only acceptable for small datasets because it requires a substantial amount of texture memory. This problem is compounded by the rapidly increasing size of datasets due to higher resolution scanners, putting a high premium on texture memory space. Furthermore, since gradients are calculated for all voxels, including those that do not contribute to the final image, per-voxel gradient computation is generally computationally inefficient.
Gradient-based shading tends to be very sensitive to noise, making it difficult to obtain high-quality visual results with noisy datasets, such as those from ultrasound and magnetic resonance (MR) scans. Additionally, gradients inside homogeneous regions tend to be unstable, having very low magnitudes and arbitrary directions. As such, they can produce shading artifacts along cutting plane boundaries, necessitating special consideration when rendering these regions.
Thus, there remains a need for producing realistic shaded images to achieve diffuse shading without requiring expensive gradient computation.